Tree-Structured Boosting: Connections Between Gradient Boosted Stumps and Full Decision Trees
نویسندگان
چکیده
José Marcio Luna 1 Eric Eaton 2 Lyle H. Ungar 2 Eric Diffenderfer 1 Shane T. Jensen 3 Efstathios D. Gennatas 4 Mateo Wirth 3 Charles B. Simone II 5 Timothy D. Solberg 4 Gilmer Valdes 4 1 Dept. of Radiation Oncology, University of Pennsylvania {Jose.Luna,Eric.Diffenderfer}@uphs.upenn.edu 2 Dept. of Computer and Information Science, University of Pennsylvania {eeaton,ungar}@cis.upenn.edu 3 Dept. of Statistics, University of Pennsylvania {stjensen,mwirth}@wharton.upenn.edu 4 Dept. of Radiation Oncology, University of California, San Francisco {Efstathios.Gennatas,Timothy.Solberg,Gilmer.Valdes}@ucsf.edu 5 Dept. of Radiation Oncology, University of Maryland Medical Center [email protected]
منابع مشابه
Obtaining Calibrated Probabilities from Boosting
Boosted decision trees typically yield good accuracy, precision, and ROC area. However, because the outputs from boosting are not well calibrated posterior probabilities, boosting yields poor squared error and cross-entropy. We empirically demonstrate why AdaBoost predicts distorted probabilities and examine three calibration methods for correcting this distortion: Platt Scaling, Isotonic Regre...
متن کاملAn Empirical Evaluation of Supervised Learning for ROC Area
We present an empirical comparison of the AUC performance of seven supervised learning methods: SVMs, neural nets, decision trees, k-nearest neighbor, bagged trees, boosted trees, and boosted stumps. Overall, boosted trees have the best average AUC performance, followed by bagged trees, neural nets and SVMs. We then present an ensemble selection method that yields even better AUC. Ensembles are...
متن کاملAn Empirical Comparison of Supervised Learning Algorithms Using Different Performance Metrics
We present results from a large-scale empirical comparison between ten learning methods: SVMs, neural nets, logistic regression, naive bayes, memory-based learning, random forests, decision trees, bagged trees, boosted trees, and boosted stumps. We evaluate the methods on binary classification problems using nine performance criteria: accuracy, squared error, cross-entropy, ROC Area, F-score, p...
متن کاملComparing ensembles of decision trees and neural networks for one-day-ahead streamflow prediction
Ensemble learning methods have received remarkable attention in the recent years and led to considerable advancement in the performance of the regression and classification problems. Bagging and boosting are among the most popular ensemble learning techniques proposed to reduce the prediction error of learning machines. In this study, bagging and gradient boosting algorithms are incorporated in...
متن کاملOptimization with Gradient-Boosted Trees and Risk Control
Decision trees effectively represent the sparse, high dimensional and noisy nature of chemical data from experiments. Having learned a function from this data, we may want to thereafter optimize the function, e.g., picking the best chemical process catalyst. In this way, we may repurpose legacy predictive models. This work studies a large-scale, industrially-relevant mixed-integer quadratic opt...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1711.06793 شماره
صفحات -
تاریخ انتشار 2017